Communicating between a virtual area and a physical space

ABSTRACT

Apparatus transmits an identifier for association with a virtual area by an administering network service, generates output data from human perceptible stimulus in a physical space, transmits the output data in connection with the virtual area, receives input data associated with the virtual area, and generates human perceptible stimulus in the physical space from the input data. A persistent association is created between the apparatus and a virtual area. A respective presence is established in the virtual area for a communicant operating a client network node connected to the virtual area. A respective connection between each active pair of complementary sources and sinks of the client network node and the apparatus are administered in association with the virtual area. A client network node displays a graphical user interface, establishes the administered connections, and presents interaction controls associated with the object for interacting with communicants in the physical space.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/437,335, filed Feb. 20, 2017, which is a continuation of U.S. patentapplication Ser. No. 14/930,472, filed Nov. 2, 2015, which is acontinuation of U.S. patent application Ser. No. 13/554,051, filed Jul.20, 2012, which claims the benefit of U.S. Provisional Application No.61/510,698, filed Jul. 22, 2011, and U.S. Provisional Application No.61/637,190, filed Apr. 23, 2012, the entirety of each of which isincorporated herein by reference.

This application also relates to the following patent applications, theentirety of each of which is incorporated herein by reference: U.S.patent application Ser. No. 13/409,344, filed Mar. 1, 2012; U.S.application Ser. No. 13/229,349, filed Sep. 9, 2011; U.S. applicationSer. No. 13/229,395, filed Sep. 9, 2011; U.S. application Ser. No.13/209,812, filed Aug. 15, 2011; U.S. application Ser. No. 12/825,512,filed Jun. 29, 2010; U.S. application Ser. No. 12/694,126, filed Jan.26, 2010; U.S. application Ser. No. 12/509,658, filed Jul. 27, 2009;U.S. application Ser. No. 12/418,243, filed Apr. 3, 2009; U.S.application Ser. No. 12/418,270, filed Apr. 3, 2009; U.S. applicationSer. No. 12/354,709, filed Jan. 15, 2009; U.S. application Ser. No.12/630,973, filed on Dec. 4, 2009; U.S. application Ser. No. 12/818,517,filed Jun. 18, 2010; U.S. patent application Ser. No. 12/855,210, filedAug. 12, 2010; U.S. Provisional Patent Application No. 61/563,088, filedNov. 23, 2011; and U.S. Provisional Patent Application No. 61/535,910,filed Sep. 16, 2011.

BACKGROUND

When face-to-face communications are not practical, people often rely onone or more technological solutions to meet their communications needs.These solutions typically are designed to simulate one or more aspectsof face-to-face communications. Traditional telephony systems enablevoice communications between callers. Instant messaging (also referredto as “chat”) communications systems enable users to communicate textmessages in real time through instant message computer clients. Someinstant messaging systems additionally allow users to be represented ina virtual environment by user-controllable graphical objects (referredto as “avatars”). Interactive virtual reality communication systemsenable users in remote locations to communicate and interact with eachother by manipulating their respective avatars in virtual spaces.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of an example of a network communicationsenvironment that includes virtual presence apparatus in a physicalspace, a remote client network node, and a virtual environment creator.

FIG. 2 shows a flow diagram of a method of communicating between avirtual area and a physical space.

FIG. 3 is a flow diagram of an example of a method performed by anexample of virtual presence apparatus.

FIG. 4 is a block diagram of an example of virtual presence apparatus.

FIG. 5A is a block diagram of an example of virtual presence apparatusconnected to a server network node.

FIG. 5B is a block diagram of an example of virtual presence apparatusconnected to a server network node.

FIG. 5C is a block diagram of an example of virtual presence apparatusconnected to a server network node.

FIG. 6 is a flow diagram of an example of a method of administeringcommunications between a virtual area and a physical space.

FIG. 7 is a flow diagram of an example of a method of communicatingbetween a virtual area and a physical space.

FIG. 8 is a diagrammatic view of an example of a graphical interface forinterfacing a user with an example of virtual presence apparatus in aphysical space.

FIG. 9 is a diagrammatic view of an example of a physical space and anexample of a graphical interface for interfacing a user with an exampleof virtual presence apparatus in the physical space.

FIG. 10 is a diagrammatic view of an example of a physical space and anexample of a graphical interface for interfacing a user with an exampleof virtual presence apparatus in the physical space.

FIG. 11 is a diagrammatic view of an example of a physical space and anexample of a graphical interface for interfacing a user with an exampleof virtual presence apparatus in the physical space.

DETAILED DESCRIPTION

In the following description, like reference numbers are used toidentify like elements. Furthermore, the drawings are intended toillustrate major features of exemplary embodiments in a diagrammaticmanner. The drawings are not intended to depict every feature of actualembodiments nor relative dimensions of the depicted elements, and arenot drawn to scale.

I. Definition of Terms

A “communicant” is a person who communicates or otherwise interacts withother persons over one or more network connections, where thecommunication or interaction may or may not occur in the context of avirtual area. A “user” is a communicant who is operating a particularnetwork node that defines a particular perspective for descriptivepurposes.

A “computer” is any machine, device, or apparatus that processes dataaccording to computer-readable instructions that are stored on acomputer-readable medium either temporarily or permanently. A “computeroperating system” is a software component of a computer system thatmanages and coordinates the performance of tasks and the sharing ofcomputing and hardware resources. A “software application” (alsoreferred to as software, an application, computer software, a computerapplication, a program, and a computer program) is a set of instructionsthat a computer can interpret and execute to perform one or morespecific tasks. A “data file” is a block of information that durablystores data for use by a software application.

The term “computer-readable medium” (also referred to as “memory”)refers to any tangible, non-transitory medium capable storinginformation (e.g., instructions and data) that is readable by a machine(e.g., a computer). Storage devices suitable for tangibly embodying suchinformation include, but are not limited to, all forms of physical,non-transitory computer-readable memory, including, for example,semiconductor memory devices, such as random access memory (RAM), EPROM,EEPROM, and Flash memory devices, magnetic disks such as internal harddisks and removable hard disks, magneto-optical disks, DVD-ROM/RAM, andCD-ROM/RAM.

A “window” is a visual area of a display that typically includes a userinterface. A window typically displays the output of a software processand typically enables a user to input commands or data for the softwareprocess. A window that has a parent is called a “child window.” A windowthat has no parent, or whose parent is the desktop window, is called a“top-level window.” A “desktop” is a system-defined window that paintsthe background of a graphical user interface (GUI) and serves as thebase for all windows displayed by all software processes.

A “data sink” (referred to herein simply as a “sink”) is any of a device(e.g., a computer), part of a device, or software that receives data.

A “data source” (referred to herein simply as a “source”) is any of adevice (e.g., a computer), part of a device, or software that originatesdata.

A “network node” (also referred to simply as a “node”) is a junction orconnection point in a communications network. Examples of network nodesinclude, but are not limited to, a terminal, a computer, and a networkswitch. A “server” network node is a host computer on a network thatresponds to requests for information or service. A “client network node”is a computer on a network that requests information or service from aserver.

A “network connection” is a link between two communicating networknodes. A “connection handle” is a pointer or identifier (e.g., a uniformresource identifier (URI)) that can be used to establish a networkconnection with a network resource. A “network communication” caninclude any type of information (e.g., text, voice, audio, video,electronic mail message, data file, motion data stream, and data packet)that is transmitted or otherwise conveyed from one network node toanother network node over a network connection.

Synchronous conferencing refers to communications in which communicantsparticipate at the same time. Synchronous conferencing encompasses alltypes of networked collaboration technologies, including instantmessaging (e.g., text chat), audio conferencing, video conferencing,application sharing, and file sharing technologies.

A “communicant interaction” is any type of direct or indirect action orinfluence between a communicant and another network entity, which mayinclude for example another communicant, a virtual area, or a networkservice. Examples of types of communicant communications includecommunicants communicating with each other in realtime, a communicantentering a virtual area, and a communicant requesting access to aresource from a network service.

“Presence” refers to the ability and willingness of a networked entity(e.g., a communicant, service, or device) to communicate, where suchwillingness affects the ability to detect and obtain information aboutthe state of the entity on a network and the ability to connect to theentity.

A “realtime data stream” is data that is structured and processed in acontinuous flow and designed to be received with no delay or onlyimperceptible delay. Realtime data streams include digitalrepresentations of voice, video, user movements, facial expressions andother physical phenomena, as well as data within the computingenvironment that may benefit from rapid transmission, rapid execution,or both rapid transmission and rapid execution, including for example,avatar movement instructions, text chat, realtime data feeds (e.g.,sensor data, machine control instructions, transaction streams and stockquote information feeds), screen shares, and file transfers.

A “physical space” is a three-dimensional real-world environment inwhich a communicant can be located physically.

A “virtual area” (also referred to as an “area” or a “place”) is arepresentation of a computer-managed space or scene. Virtual areastypically are one-dimensional, two-dimensional, or three-dimensionalrepresentations; although in some examples a virtual area may correspondto a single point. Oftentimes, a virtual area is designed to simulate aphysical, real-world space. For example, using a traditional computermonitor, a virtual area may be visualized as a two-dimensional graphicof a three-dimensional computer-generated space. However, virtual areasdo not require an associated visualization. A virtual area typicallyrefers to an instance of a virtual area schema, where the schema definesthe structure and contents of a virtual area in terms of variables andthe instance defines the structure and contents of a virtual area interms of values that have been resolved from a particular context.

A “persistent virtual area” is a virtual area that persists even afterall communicants have disconnected from the virtual area. The state of apersistent virtual area is preserved so that it can be restored the nexttime a communicant connects to the virtual area. A “persistentassociation” between a virtual area and virtual presence apparatus is anassociation that persists even after all communicants and the virtualpresence apparatus have disconnected from the virtual area.

A “virtual area application” (also referred to as a “virtual areaspecification”) is a description of a virtual area that is used increating a virtual environment. A virtual area application typicallyincludes definitions of geometry, physics, and realtime switching rulesthat are associated with one or more zones of the virtual area.

A “virtual area enabled communications application” is a clientcommunications application that integrates realtime communications(e.g., synchronous conferencing functionalities, such as audio, video,chat, and realtime other data communications) with a virtual area.

A “virtual environment” is a representation of a computer-managed spacethat includes at least one virtual area and supports realtimecommunications between communicants.

A “position” in a virtual area refers to a location of a point or anarea or a volume in the virtual area. A point typically is representedby a single set of one-dimensional, two-dimensional, orthree-dimensional coordinates (e.g., x, y, z) that define a spot in thevirtual area. An area typically is represented by the three-dimensionalcoordinates of three or more coplanar vertices that define a boundary ofa closed two-dimensional shape in the virtual area. A volume typicallyis represented by the three-dimensional coordinates of four or morenon-coplanar vertices that define a closed boundary of athree-dimensional shape in the virtual area.

VoIP (Voice over Internet Protocol) refers to systems and methods ofdelivering voice and other communications over Internet Protocol (IP)networks.

As used herein, the term “includes” means includes but not limited to,the term “including” means including but not limited to. The term “basedon” means based at least in part on.

II. Communicating Between a Virtual Area and a Physical Space

Examples that are described herein provide systems and methods ofcommunicating between a virtual area and a physical space. Theseexamples bridge the virtual area into the physical space and bridge thephysical space into the virtual area through virtual presence apparatus(VPA) located in the physical space. Examples of the virtual presenceapparatus transduce human perceptible stimulus (e.g., audio, visual,mechanical, and other sensory stimulus) between the virtual area and thephysical space such that communicant interactions in the virtual areaare seamlessly integrated into the physical space and vice versa.

FIG. 1 shows an embodiment of an exemplary network communicationsenvironment 10 that includes a virtual presence apparatus 12 in aphysical space 14, a remote client network node 16, and a virtualenvironment creator 18 that are interconnected by a network (not shown)that supports the transmission of a wide variety of different mediatypes (e.g., text, voice, audio, video, and other data) between networknodes. The network connections between network nodes may be arranged ina variety of different stream handling topologies, including apeer-to-peer architecture, a server-mediated architecture, and hybridarchitectures that combine aspects of peer-to-peer and server-mediatedarchitectures. Exemplary topologies of these types are described in U.S.Pat. Nos. 7,769,806 and 7,844,724.

The client network node 16 includes input/output (I/O) hardware, aprocessor, and a computer-readable memory that stores an instance 20 ofat least one virtual area enabled communications application that isexecutable by the processor. The communications application 20 typicallyprovides graphical interface and communications functions forcommunicating with the virtual presence apparatus 12, the virtualenvironment creator 18, and other client network nodes in connectionwith one or more virtual areas. Examples of the communicationsapplications are described in U.S. application Ser. No. 12/418,243,filed Apr. 3, 2009, U.S. application Ser. No. 12/630,973, filed Dec. 4,2009, U.S. application Ser. No. 12/354,709, filed Jan. 15, 2009, U.S.application Ser. No. 12/509,658, filed Jul. 27, 2009, U.S. applicationSer. No. 13/209,812, filed Aug. 15, 2011, and U.S. application Ser. No.13/229,349, filed Sep. 9, 2011. The client network node 16 has arespective set of one or more sources and a respective set of one ormore sinks. Exemplary sources include an audio source (e.g., an audiocapture device, such as a microphone), a video source (e.g., a videocapture device, such as a video camera), a chat source (e.g., a textcapture device, such as a keyboard), a motion data source (e.g., apointing device, such as a computer mouse), and other sources (e.g.,file sharing source or a source of a customized real-time data stream).Exemplary sinks include an audio sink (e.g., an audio rendering device,such as a speaker or headphones), a video sink (e.g., a video renderingdevice, such as a display monitor), a chat sink (e.g., a text renderingdevice, such as a display monitor), a motion data sink (e.g., a movementrendering device, such as a display monitor), and other sinks (e.g., aprinter for printing shared files, a device for rendering real-time datastreams different from those already described, or software thatprocesses real-time streams for analysis or customized display). Theclient network node 16 also typically includes administrative policies,user preferences (including preferences regarding the exportation of theuser's presence and the connection of the user to the virtualenvironment creator 18 and other communicants), and other settings thatdefine a local configuration that influences the administration ofrealtime connections with the virtual presence apparatus 12, the virtualenvironment creator 18, and other network nodes.

The virtual presence apparatus 12 is located in the physical space 14.In the illustrated example, the virtual presence apparatus 12 ispositioned on a table 22 in a real-world conference room containing fivecommunicants 24, 26, 28, 30, 32. The four communicants 24-30 are seatedaround the table 22, and the fifth communicant 32 is standing beside areal-world view screen 34. Three of the seated communicants 24-28 areoperating respective virtual area enabled communications applications ontheir client network nodes 36, 38, 40 (e.g., mobile computers, such aslaptop computers, tablet computers, and mobile phones) through whichthey are connected to the remote client network node 16; these threecommunicants 24-28 and the other two communicants 30, 32 are connectedto the remote client network node 16 through the virtual presenceapparatus 12.

The virtual presence apparatus 12 typically includes software andhardware resources that enable the virtual presence apparatus 12 toconnect to the virtual environment creator 18 and the remote clientnetwork node 16, either directly (e.g., peer-to-peer) or through ahosted network connection. In some examples, the virtual presenceapparatus 12 or a network node hosting the virtual presence apparatusincludes a complete or modified version of the communicationsapplication 20, which provides functions for communicating with thevirtual environment creator 18 and establishing network connections andcommunicating realtime data streams with the client network nodes. Whenconnected to the virtual environment creator 18, the virtual presenceapparatus 12 can be registered in association with and/or logged intothe one or more virtual areas. When logged into a virtual area, thevirtual presence apparatus 12 transduces human perceptible stimulus(e.g., audio, visual, mechanical, and other sensory stimulus) betweenthe client network nodes of communicants who are present in the virtualarea and the physical space 14. In this way, the virtual presenceapparatus 12 bridges a physical experience of the physical space 14 tocommunicants in the one or more virtual areas (i.e., communicants whoare present in the virtual areas) and bridges communicant interactionsin the one or more virtual areas to communicants in the physical space14.

In the illustrated example, the virtual environment creator 18 includesat least one server network node 42 that provides a networkinfrastructure service environment 44 that manages sessions of theremote client network node 16 and the virtual presence apparatus 12 inone or more virtual areas 46 in accordance with respective virtual areaapplications 48. Each of the virtual area applications 48 is hosted by arespective one of the virtual areas 46 and includes a description of therespective virtual area 46. Communicants operating respective clientnetwork nodes connect to the virtual area applications 48 throughvirtual area enabled communications applications.

A virtual area typically includes one or more zones. A zone may be arendered spatial extent, a set of rules applied to a spatial extent, orboth. Zones may be arranged hierarchically in a virtual area, with anoutermost zone (referred to herein as the “Global Governance Zone”)enclosing all other zones in the virtual area. Within the GlobalGovernance Zone, there can be location zones (e.g., rooms of a virtualarea) or smaller governance zones that enclose a group of location zonesand provide regions of governance on the map. A zone definitiontypically also includes one or more channel definitions that describehow to create respective channels in the zone and specify theinformation about the channel that is published to a client network nodethat becomes present in the zone. A channel is always uniquely definedpoint-to-point and is unique to a virtual area application and a sessionbetween a client network node and the virtual area platform.

Examples of the types of rules that may be associated with a zoneinclude switching rules, governance rules, and permission rules.

Switching rules govern realtime stream connections between network nodesthat are linked to the virtual area (e.g., network nodes that areassociated with objects, such as avatars, in the virtual area). Theswitching rules typically include a description of conditions forconnecting sources and sinks of realtime data streams in terms ofpositions in the virtual area. Each switching rule typically includesattributes that define the realtime data stream type to which the ruleapplies and the location or locations in the virtual area where the ruleapplies. In some examples, each of the rules optionally may include oneor more attributes that specify a required role of the source, arequired role of the sink, a priority level of the stream, and arequested data routing topology. In some examples, if there are noexplicit switching rules defined for a particular part of the virtualarea, one or more implicit or default switching rules may apply to thatpart of the virtual area.

Governance rules control who has access to resources (e.g., the virtualarea itself, regions with the virtual area, and objects within thevirtual area), who has access to data (e.g., data streams and othercontent) that is associated with the virtual area, what is the scope ofthat access to the data associated the virtual area (e.g., what can auser do with the data), and what are the follow-on consequences ofaccessing that data (e.g., record keeping, such as audit logs, andpayment requirements). In some examples, an entire virtual area or azone of the virtual area is associated with a “governance mesh” thatenables a software application developer to associate governance ruleswith a virtual area or a zone of a virtual area. This avoids the needfor the creation of individual permissions for every file in a virtualarea and avoids the need to deal with the complexity that potentiallycould arise when there is a need to treat the same document differentlydepending on the context.

A permission rule defines a respective capability requirement (e.g., fora respective action, behavior, or state) in terms of one or morecapabilities, attributes, and settings, which may be persistent ortransient. Examples of capabilities systems for administering permissionrules are described in U.S. Provisional Patent Application No.61/535,910, filed Sep. 16, 2011.

In some examples, a virtual area is defined by a specification thatincludes a description of geometric elements of the virtual area and oneor more rules, including switching rules and governance rules. Examplesof virtual area specifications are described in U.S. application Ser.No. 12/418,243, filed Apr. 3, 2009, U.S. application Ser. No.12/818,517, filed Jun. 18, 2010, U.S. patent application Ser. No.12/855,210, filed Aug. 12, 2010, and U.S. Provisional Application No.61/563,088, filed Nov. 23, 2011.

The network infrastructure service environment 44 typically includes oneor more network infrastructure services that cooperate with the virtualarea enabled communications application 20 to establish and administernetwork connections between the virtual presence apparatus 12, theremote client network node 16, and other network nodes. Among thenetwork infrastructure services that are included in an exemplaryembodiment of the network infrastructure service environment 44 are anaccount service, a security service, an area service, a rendezvousservice, and an interaction service. The structure, operation, andcomponents of such an embodiment of the network infrastructure serviceenvironment 44 are described in U.S. patent application Ser. No.12/825,512, filed Jun. 29, 2010.

The virtual area enabled communications applications 20, the areaapplications 48, and the network infrastructure service environment 44together provide a platform (referred to herein as “the platform”) thatadministers the realtime connections with network nodes in a virtualarea subject to a set of constraints that control access to the virtualarea instance.

The platform tracks communicants' realtime availabilities and activitiesacross the different communication contexts that are defined by the areaapplications 48. This information is presented by the virtual areaenabled communications applications to the communicants in the form ofrealtime visualizations that enable the communicants to make moreinformed network interaction decisions (e.g., when to interact with acontact) and encourages the communicants to initiate interactions withother communicants and to join contexts (e.g., an ongoing conversationbetween communicants) of which the communicants otherwise would not havebeen aware. In some embodiments, the realtime visualization includesvisual cues as to the presence and activities of the communicants in thecontexts of the area applications 48. The presentation of these visualcues typically depends on one or more of governance rules associatedwith the virtual areas 46, administrative policies, and user preferences(including preferences regarding the exportation of the user's presenceand the connection of the user to areas and other communicants), whichmay define tiered relationship based predicates that control access topresence information and/or network resources on a zone-by-zone basis.

In some embodiments, the server network node 42 remotely manages clientcommunication sessions with each other and with the virtual presenceapparatus 12, and remotely configures audio and graphic renderingengines on the client network nodes, as well as switching of datastreams by sending instructions (also referred to as definitions) fromthe remotely hosted area applications 48 to the client network nodes inaccordance with the stream transport protocol described in U.S.application Ser. No. 12/825,512, filed Jun. 29, 2010. Data is sharedbetween the client network node 16 and other network nodes as definitionrecords over transport protocol sockets. The client communicationsapplication 16 receives configuration instructions from the server node42 through definition records that are received over a server sessionbetween the client network node 16 and the server network node 42. Insome of these examples, the server network node 42 sends to each of theclient network nodes provisioning messages that configure the clientnetwork nodes to interconnect respective data streams between activeones of their complementary sources and sinks over respectivepeer-to-peer (P2P) sessions in accordance with switching rules specifiedin the area applications 48 and the locations where the communicants andthe virtual presence apparatus are present in the virtual area 46. Theclient network node 16 sends content to and receives content from othernetwork nodes through definition records that are transmitted oncontent-specific channels on respective sessions with the other networknodes. Data is shared in accordance with a publish/subscribe model. Astream transport service on the client network node 16 subscribes onlyto the data that are needed by the client network node. To subscribe,the stream transport service negotiates a channel on a session that isestablished with another network node. The channel is negotiated bywell-known GUID for the particular area application 48. Definitionrecords are transmitted only when a subscriber exists on the other endof a transport protocol socket. Definition records that are received bythe stream transport service are delivered to the subscribing ones ofthe client communications application processes on arrival. In this way,the server network node 42 connects the virtual presence apparatus 12 tothe virtual area 46 so that the virtual presence apparatus 12 can bridgea physical experience of the physical space 14 to communicants in thevirtual area 46 and bridge a physical experience of communicantinteractions in the virtual area 46 to communicants in the physicalspace 14.

In the illustrated embodiment, the communications application 20operating on the remote client network node 16 presents a respectivespatial visualization 50 of the virtual area 46 in accordance with datareceived from the network infrastructure service environment 44. Thecommunications application 20 also provides a graphical interface forreceiving user commands and providing a spatial interface that enhancesthe realtime communications between the communicants. The spatialvisualization 50 includes respective graphical representations 52, 54,56, 58 (referred to herein as “avatars” or “sprites”) of thecommunicants who are present in the virtual area 46 in spatial relationto a graphical representation 59 of the virtual area 46. In theillustrated example, the sprites 52, 54, 56 represent the threecommunicants 24, 26, 28 (Beth, Fran, Art) who are seated in the physicalspace 14 and are operating the local client network nodes 36, 38, 40,and the sprite 58 represents the communicant (Ed) who is operating theremote client network node 16. The spatial visualization 50 may includeother objects (also referred to as “props”). Examples of such objectsinclude a view screen object 60 for interfacing with application sharingfunctions of the communications application 20 (as described in, e.g.,U.S. application Ser. No. 12/418,270, filed Apr. 3, 2009), a tableobject 62 for interfacing with file sharing functions of thecommunications application 20, and a VPA object 64 for interfacing withthe virtual presence apparatus 12 in the physical space 14. The spatialvisualization 50 typically is presented in a respective window 66 thatis generated by the communications application 20 on a “desktop” orother system-defined base window on the display hardware 68 of theremote client network node 16.

The activities of the communicants in the virtual area 46 may beinferred from the activities on the various communication channels overwhich the respective client network nodes are configured to communicate.The activities on the communication channels are represented in thegraphical interface by visual cues that are depicted in association withthe graphical representations 52-58 of the communicants. For example,the “on” or “off” state of a communicant's local speaker channel isdepicted by the presence or absence of a headphones graphic 61 on thecommunicant's sprite. When the speakers of the communicant who isrepresented by the sprite are on, the headphones graphic 61 is present(see sprites Beth and Fran) and, when the communicant's speakers areoff, the headphones graphic 61 is absent. The “on” or “off” state of thecommunicant's microphone is depicted by the presence or absence of amicrophone graphic 63 on the communicant's sprite. When the microphoneis on, the microphone graphic 63 is present (see sprite Fran); and, whenthe microphone is off, the microphone graphic 63 is absent. Theheadphones graphic 61 and the microphone graphic 63 provide visual cuesof the activity states of the communicant's sound playback andmicrophone devices. The “on” or “off” state of the communicant'smicrophone is depicted by the presence or absence of a microphonegraphic on the communicant's graphic representation and a series ofconcentric circles 65 that dynamically radiate away from thecommunicant's graphic representation in a series of expanding waves.When the microphone is on, the microphone graphic 63 and the radiatingconcentric circles 65 are present and, when the microphone is off, themicrophone graphic 63 and the radiating concentric circles 65 areabsent. In addition to or alternatively, the current activity on acommunicant's microphone channel is indicated by a dynamic visualizationthat lightens and darkens the communicant's avatar in realtime toreflect the presence or absence of audio data on the microphone channel.Thus, whether or not their local speakers are turned on, communicantscan determine when another communicant is speaking by the “blinking” ofthe coloration of that communicant's avatar.

The activity on a communicant's text chat channel is depicted by thepresence or absence of the hand graphic 67 adjacent the communicant'ssprite (see sprite Ed). Thus, when a communicant is transmitting textchat data to another network node the hand graphic 67 is present, andwhen a communicant is not transmitting text chat data the hand graphic67 is not present. In some embodiments, text chat data is transmittedonly when keyboard keys are depressed, in which case the visualizationof the communicant's text channel appears as a flashing on and off ofthe hand graphic 67.

The view screen prop 60 is associated with application sharingfunctionality of the platform that enables communicants to shareapplications operating their respective client network nodes. Theapplication sharing functionality is invoked by activating a view screen(e.g., by single-clicking the view screen object with an input device).In some embodiments, the platform provides visual cues that indicatewhether or not a communicant is sharing an application over anapplication sharing channel. In response to a communicant's selection ofthe view screen prop, the communicant's sprite automatically is moved toa position in the graphical representation of the virtual area that isadjacent the view screen prop. The position of a communicant's spriteadjacent the view screen prop indicates that the communicant currentlyis sharing or is about to share an application with the othercommunicants in the virtual area. Other communicants in the virtualsubscribe to the shared application data by selecting the view screenprop in their respective views of the spatial visualization 50. Theavatar of each communicant who is viewing a shared application isdepicted with a pair of “eyes” to indicate that the representedcommunicants are viewing the content being shared in connection with theview screen props. The graphical depiction of view screen prop ischanged depending on whether or not an active application sharingsession is occurring. For example, the depicted color of the view screenmay change from a brighter color during an active application sharingsession to a darker color when there is no application sharing takingplace. Examples of the application sharing process are described inconnection with FIGS. 26-28 of U.S. patent application Ser. No.12/354,709, filed Jan. 15, 2009, and in U.S. patent application Ser. No.12/418,270, filed Apr. 3, 2009.

FIG. 2 shows an example of a method by which the virtual environmentcreator 18 and the virtual presence apparatus 12 bridge the virtual areainto the physical space and bridge the physical space into the virtualarea. In accordance with this method, the virtual presence apparatus 12transforms human perceptible physical stimuli in a physical space intophysical space data streams of different respective data stream types(FIG. 2, block 101). The server network node 42 publishes respectiveones of the physical space data streams in a zone of a virtual area in avirtual communications environment (FIG. 2, block 103). The zone definesa respective persistent context for realtime communications betweennetwork nodes of respective communicants who are present in the zone.The server network node 42 establishes a respective presence in the zonefor each of multiple communicants associated with respective clientnetwork nodes. Each of one or more of the client network nodes publishesone or more respective client data streams (FIG. 2, block 105). Theserver network node 42 provisions data stream connections fortransmitting respective ones of the published physical space datastreams to respective ones of the client network nodes, transmittingrespective ones of the published client data streams to respective onesof the client network nodes, and transmitting respective ones of thepublished client data streams to the physical space (FIG. 2, block 107).The virtual presence apparatus 12 transforms the published client datastreams transmitted to the physical space into human perceptiblephysical stimuli in the physical space (FIG. 5, block 109).

FIG. 3 shows an example of a process that is implemented by an exampleof the virtual presence apparatus 12. In accordance with this method,the virtual presence apparatus 12 transmits a globally unique identifierof the virtual presence apparatus 12 for association with a virtual areaby a network service administering the virtual area (FIG. 3, block 90).The virtual presence apparatus 12 generates output data from humanperceptible stimulus in a physical space (FIG. 3, block 92), andtransmits the output data in connection with the virtual area (FIG. 3,block 94). The virtual presence apparatus 12 receives input dataassociated with the virtual area (FIG. 3, block 96), and generates humanperceptible stimulus in the physical space from the input data (FIG. 3,block 98).

FIG. 4 shows an example 70 of the virtual presence apparatus 12 thatincludes an input transducer 72, an output transducer 74, acommunication interface 76, a computer-readable memory 78 that stores aglobally unique identifier of the virtual presence apparatus 70, and aprocessor 80. The communication interface 76 transmits an output signal86 and receives an input signal 88.

The virtual presence apparatus 70 may be implemented in a variety ofdifferent ways. In some examples, the virtual presence apparatus 70 iscomposed of multiple components (e.g., two or more of a speaker, amicrophone, a light projector, and a camera) that are integrated into aunitary device. In other examples, the virtual presence apparatus 70 iscomposed of a central hub (e.g., a virtual area enabled network switchor router) that controls and configures one or more separate anddistinct peripheral components (e.g., a speakerphone, a digitalprojector, a camera, and a remote-controlled laser pointer) that areconnected to respective ports (e.g., Universal Serial Bus (USB) ports)of the hub. Examples of the virtual presence apparatus 70 may havedifferent industrial designs. In some examples, the virtual presenceapparatus 70 has the form factor of a desktop appliance (e.g., a formfactor similar to that of a computer, speakerphone, a digital projector,or a network hub), whereas other examples of the virtual presenceapparatus 70 have robotic form factors (e.g., a remote-controlledelectro-mechanical machine, which may or may not have a humanoidappearance).

The input transducer 72 generates output data from human perceptiblestimulus 82 in the physical space 14. The input transducer 72 typicallygenerates the output data from human perceptible stimulus that isbroadcasted into the physical space. Depending on the desiredcommunication application, the input transducer 72 may generate outputdata from one or more human perceptible stimuli, including for exampleaudio, visual, mechanical, and other sensory stimuli. In some examples,the input transducer 72 includes one or more of an acoustic-to-electrictransducer (e.g., a microphone, which may be a component of a telephonydevice, such as a mobile phone or a VoIP phone, or a headset), alight-to-electric transducer (e.g., a camera, such as a still imagecamera, a video camera, and a scanner that scans physical documents intoscanned images), an electric-to-electric transducer (e.g., a touchscreenor other touch-sensitive sensor equipped with resistive, capacitive,surface acoustic wave, optical, or other touch-sensitive technologies),a mechanical-to-electric transducer (e.g., a tactile or other pressure-or force-sensitive transducer, a texture-sensitive transducer), and achemical-to-electric transducer (e.g., a olfactory sensor that iscapable of detecting one or more odorants).

The output transducer 74 generates human perceptible stimulus 84 in thephysical space 14. The output transducer 74 typically broadcasts thehuman perceptible stimulus into the physical space. Depending on thedesired communications application, the output transducer 74 maygenerate one or more human perceptible stimuli from input data,including for example audio, visual, mechanical, and other sensorystimuli. In some examples, the output transducer 74 includes one or moreof an electric-to-acoustic transducer (e.g., a speaker, which may be acomponent of a telephony device, such as a mobile phone or a VoIP phone,or a headset), an electric-to-light transducer (e.g., an image projectorsuch as a digital projector, a touchscreen display, a light beamprojector such as a laser pointer, or a three-dimensional hologramgenerator), an electric-to-mechanical transducer (e.g., a haptictransducer, an electric motor that moves mechanical components, such aslight sources and robot tools, and other components in the physicalspace, and a printer that outputs printed documents or three-dimensionalobjects), and an electric-to-chemical transducer (e.g., an electricodorant delivery system).

The virtual presence apparatus 70 bridges communicant activity in thephysical space 14 into the virtual area 46 and bridges communicantactivity in the virtual area into the physical space 14. In thisprocess, the virtual presence apparatus 70 typically encodes output datagenerated by the input transducer 72 from communicant activity in thephysical space 14 into the output signal 86 that is sent to the remotenetwork node 16 connected to the virtual area, and decodes the inputsignal 88, which is received from the remote network node 16 and relatesto communicant activity in the virtual area, into input data that issent to the output transducer 74.

The virtual presence apparatus 12 typically is registered with theserver network node 42 before the virtual presence apparatus 12 can belogged into a server session with the server network node 42. In someexamples, the virtual presence apparatus 12 includes hardware andsoftware resources that enable it to register directly with the servernetwork node 42.

For example, FIG. 5A shows an example of a network connection betweenthe server network node 42 and an example 85 of the virtual presenceapparatus 12 that can register directly with the server network node 42.

In other examples, a host computer (e.g., one of the client networknodes 36-40 in the physical space) registers the virtual presenceapparatus 12 with the server network node 42. FIG. 5B shows an exampleof a network connection between the server network node 42 and anexample 87 of the virtual presence apparatus 12 that is hosted by aclient network node 89 in the physical space 14. In this example, theclient network node 89 submits to the server network node 42registration information and login requests on behalf of both thevirtual presence apparatus 87 and the communicant who uses the clientnetwork node 89 to access the virtual area 46.

FIG. 5C shows an example of a network connection between the servernetwork node 42 and an example of 93 of the virtual presence apparatus12 that includes one or more integral components of a client networknode 91 in the physical space 14. The virtual presence apparatus 93typically includes one or more hardware and software resources of theclient network node 91. In some examples, the virtual presence apparatus93 includes software that resides in the memory of the client networknode 91 and is executed by the processor of the client network node 91to leverage hardware resources of the client network node 91 in theprocess of integrating communicant interactions in the virtual area intothe physical space. In some of these examples, hardware resources of theclient network node 91 are partitioned (e.g., by a hypervisor or virtualmachine monitor that reserves a respective set of client systemresources for each partition or virtual machine) into a set of hardwareresources that are used by the virtual area enabled communicationsapplication 20 and a separate set of hardware resources that constituteelements of the virtual presence apparatus 93. For example, a peripheralheadset may be reserved for use by the virtual area enabledcommunications application 20, whereas separate microphone and speakerhardware may be reserved for use by the virtual presence apparatus 93.In some examples, certain hardware resources of the client network node91 (e.g., a camera, a hard drive memory, or an optical disk drive) thatare allocated to the virtual presence apparatus 93 are associated withrespective objects in the virtual area 46, allowing those resources tobe shared by other communicants in the virtual area 46.

During registration, the virtual presence apparatus 12 transmits (eitherdirectly, or indirectly through a network node hosting the virtualpresence apparatus 12) registration data through its communicationinterface to the server network node 42. The registration data typicallyincludes the globally unique identifier of the virtual presenceapparatus 12 and configuration data. The configuration data may include,for example, a device type identifier, an indication whether the virtualpresence apparatus 12 should be associated with an existing virtual areaor a new virtual area, one or more conditions on the availability of theassociated virtual area (e.g., the associated virtual area is accessibleto communicants conditioned on the virtual area apparatus 12 beingpresent in or logged into the virtual area), a specification of thesource and sink capabilities of the virtual presence apparatus 12, and aspecification of a graphical representation of the virtual presenceapparatus 12. Based on the registration data, the server network node 42generates one or more database records that store the registrationinformation, including the identifier of the virtual presence apparatus12 and an identifier of the new or existing virtual area. The one ormore database records create a persistent association between thevirtual presence apparatus 12 and the virtual area. The virtual presenceapparatus identifier typically is registered with the server networknode 42 independently of any communicant identifier. The server networknode 42 determines the source and sink capabilities of the virtualpresence apparatus, either directly (e.g., from the configuration data)or indirectly (e.g., by using the device type identifier for the virtualpresence apparatus 12 as an index into a device capabilities table).

In some examples, the virtual presence apparatus 12 is associated with avirtual area independently of any particular communicant such that it isavailable as a resource for any communicant who is present in thevirtual area. In this way, the virtual presence apparatus functions as aprop or a fixture of the associated virtual area, which is tied to thephysical location of the virtual presence apparatus. In some examples,the association between the virtual presence apparatus and the virtualarea is such that the virtual area is inaccessible until after thevirtual presence apparatus has been logged into the networkinfrastructure service environment 44. In some of these examples,communicants cannot establish a presence in the associated virtual area(and, in some cases, may not even be presented with an option forentering the virtual area such that the virtual area does not appear toexist) until after the virtual presence apparatus has been connected tothe virtual area by the network infrastructure service environment 44.These examples allow communicants to establish a persistent associationbetween a virtual area and a particular physical space by leaving thevirtual presence apparatus in the same physical space, therebyleveraging the persistent spatial association with the real-worldlocation of the physical space to further strengthen the bridgingbetween the virtual area and the physical space.

After the virtual presence apparatus 12 has been registered with theserver network node 42, the virtual presence apparatus 12 can be loggedinto the network infrastructure service environment 44. The virtualpresence apparatus 12 can either log itself into the networkinfrastructure service environment 44 automatically each time it isturned on or it can be logged into the network infrastructure serviceenvironment 44 by a host computer. If the associated virtual areaalready has been instantiated, the server network node 42 sendsprovisioning instructions for establishing respective sessions betweenthe virtual presence apparatus 12 and the client network nodes of thecommunicants who are present in the virtual area and for updating theappearance of the virtual area to include a graphical representation ofthe virtual presence apparatus 12 in the graphical interfaces displayedon the client network nodes. If the associated virtual area has not yetbeen instantiated, the server network node 42 instantiates theassociated virtual area so that communicants operating respective clientnetwork nodes can access the virtual area.

The provisioning instructions sent by the server network node 42 areused to establish communication sessions between the client networknodes and the virtual presence apparatus. In some examples, data isshared between the client network nodes and the virtual presenceapparatus 12 as definition records over transport protocol sockets. Theclient network nodes and the virtual presence apparatus 12 receivecontent from each other through definition records that are received oncontent-specific channels on respective peer-to-peer sessions. Data isshared in accordance with a publish/subscribe model. A stream transportservice on each of the client network nodes and the virtual presenceapparatus 12 subscribes only to the data that are needed. To subscribe,the stream transport service negotiates a channel on a session that isestablished with another network node. The channel is negotiated bywell-known GUID for the particular area application 48. Definitionrecords are transmitted only when a subscriber exists on the other endof a transport protocol socket. Definition records that are received bythe stream transport service are delivered to the subscribing ones ofthe local communication processes on arrival. Example of the structureand operation of the stream transport service and the data sharingcommunication sessions are described in U.S. patent application Ser. No.12/825,512, filed Jun. 29, 2010.

In some examples, the virtual presence apparatus 12 transmits the outputdata corresponding to the human perceptible stimulus in the physicalspace to the client network nodes in connection with the virtual area.In this process, the virtual presence apparatus 12 typically processesthe output data and configures its communication interface toincorporate the output data into the output signal that is sent to aclient network node. In some examples, the output signal includes atleast one of the globally unique identifier of the virtual presenceapparatus, an identifier of the virtual area, and optionally anidentifier of a zone of the virtual area. The output signal typically isfree of any communicant identifier (i.e., an identifier that identifiesa particular communicant). In these examples, the virtual area (or thespecified zone of the virtual area) serves as a termination point forone or more data streams that represent physical stimuli in a physicalspace occupied by the virtual presence apparatus 12, where the datastreams are published by the virtual presence apparatus in the virtualarea/zone, communicants who are present in the virtual area/zonerespectively are able to subscribe to one or more of the data streams,and the server network node 42 provisions data stream connections forthe data streams that are subscribed to by respective ones of thecommunicants who are present in the virtual area/zone.

The virtual presence apparatus 12 typically determines the input datafrom the input signal that is received through its communicationinterface from a respective client network node that is connected to thevirtual area. The input signal typically includes at least one of aglobally unique identifier of the respective client network node and anidentifier of the virtual area. The virtual presence apparatus 12typically derives input data from the input signal and passes the inputdata to an output transducer, which generates human perceptible stimulusin the physical space.

FIG. 6 shows an example of a method that is implemented by an example ofthe server network node 42 for administering communications between avirtual area and a physical space. In accordance with this method, theserver network node 42 creates a persistent association between virtualpresence apparatus in a physical space and a virtual area (FIG. 6, block100). The apparatus has an apparatus source of a respective data streamcontent type and an apparatus sink of a respective data stream contenttype. The server network node 42 establishes a respective presence inthe virtual area for a communicant operating a client network nodeconnected to the virtual area (FIG. 6, block 102). The client networknode has a client sink that is complementary to the apparatus source anda client source that is complementary to the apparatus sink. The servernetwork node 42 administers a respective connection between each activepair of complementary sources and sinks of the client network node andthe apparatus in association with the virtual area, where eachconnection supports transmission of the respective data stream contenttype between the apparatus and the client network node (FIG. 6, block104).

In some examples of the method of FIG. 6, the association between thevirtual presence apparatus 12 and the virtual area is independent of anyparticular communicant. The server network node 42 typically receives aglobally unique identifier of the virtual presence apparatus 12, andassociates the identifier with the virtual area. In some examples, thevirtual area includes multiple zones each of which supportsestablishment of a respective presence for one or more communicants anddefines a respective persistent context for realtime communicationsbetween the client network nodes of communicants who are present in thezone. In some of these examples, the server network node 42 creates apersistent association between the physical presence apparatus and arespective one of the zones of the virtual area.

The server network node 42 establishes a respective presence in the zonefor the virtual presence apparatus. In some examples, the server networknode 42 establishes the presence for the virtual presence apparatus inresponse to receipt of a login request identifying the virtual presenceapparatus. The virtual presence apparatus or a network node (e.g., acentral hub or a client network node) that is hosting the virtualpresence apparatus may generate the login request for the virtualpresence apparatus. In some examples, the server network node 42establishes a presence for both the virtual presence apparatus and acommunicant in response to respective login requests that are sent bythe same client network node. In some examples, in response to receiptof a login request that includes the identifier of the virtual presenceapparatus, the server network node 42 initiates the virtual area toenable the virtual area to be communicant accessible.

The server network node 42 typically associates the virtual presenceapparatus 12 with an object in the virtual area. The server network node42 typically creates an object that represents the virtual presenceapparatus 12 in the virtual area. The object typically is associatedwith an interface for interacting with the virtual presence apparatus12. In some examples, the server network node 42 associates the objectwith a graphical representation of the virtual presence apparatus 12. Insome examples, the graphical representation of the virtual presenceapparatus 12 includes a brand that is associated with the virtualpresence apparatus. The brand may include a name, term, design, symbol,or any other feature that identifies a source (e.g., manufacturer orseller) of the virtual presence apparatus. The server network node 42transmits to each of the communicants who are present in the zone arespective specification of a visualization of graphical representationsof the object and the avatars in the virtual area. The client networknodes use the specifications to display respective graphicalrepresentations of the virtual presence apparatus 12 and thecommunicants in spatial relation to a graphical representation of thevirtual area.

In some examples, the object representing the virtual presence apparatusis associated with a particular communicant and the visualization of thevirtual area shows an association between a graphical representation ofthe object and the particular communicant. In some of these examples,the visualization of the virtual area shows the graphical representationof the object associated with a graphical representation of the avatarrepresenting the particular communicant. For example, the virtualpresence apparatus may be personal gear (e.g., a human interface device,such as a headset, or other personal device) that is carried or worn bythe particular communicant, and the visualization may show a graphicalrepresentation of the gear as a decoration or embellishment on thegraphical representation of the particular communicant's avatar (e.g.,showing a graphical representation of a headset on the communicant'savatar). In some examples, the visualization of the virtual area showsthe graphical representation of the object representing the virtualpresence apparatus associated with a location in the virtual area thatis assigned to the particular communicant. For example, the virtualpresence apparatus may be personal gear (e.g., a personal printer,scanner, telephony device, or a memory resource of a personal computer)that is assigned or belongs to the particular communicant, and thevisualization may show a graphical representation of the in a room(e.g., an office or personal room) of the virtual area that is assignedto the particular communicant. In some of these examples, the servernetwork node 42 may determine the style used to represent the personalgear in the visualization based on configuration information receivedfrom the particular communicant (e.g., an indication that the graphicalrepresentation of the gear should be associated with the communicant'savatar or the communicant's designated default zone, such as thecommunicant's home zone or office) or automatically based on apredefined mapping between personal gear category types and presentationstyles (e.g., headsets are represented as graphical decorations on therespective communicants' avatars, whereas hard drive of personalcomputers are represented as icons in the respective communicants'designated default zones).

In some examples, the server network node 42 transmits to the clientnetwork node a specification of visual cues for displaying indicationsof respective states of a source of the virtual presence apparatus 12.Based on a determination that that the source of the virtual presenceapparatus is in an active state, the server network node 42 transmits tothe client network node a specification of a first visual cue, and basedon a determination that the source of the virtual presence apparatus isin an inactive state, the server network node 42 transmits to the clientnetwork node a specification of a second visual cue that is differentfrom the first visual cue. In some examples, the specifications of thefirst and second visual cues are provided in respective definitionrecords.

The server network node 42 administers realtime communications betweenthe respective network nodes of the communicants who are present in thezone and provisions at least one data stream connection between thevirtual presence apparatus 12 and one or more of the network nodes ofthe communicants who are present in the zone. In some examples, theserver network node 42 administers respective connections between eachactive pair of complementary sources and sinks of the client networknode and the apparatus. These connections bridge communicant activity inthe physical space into the virtual area and bridge communicant activityin the virtual area into the physical space. In some of these examples,the server network node 42 administers connections that relay datacorresponding to communicant activity in the physical space from thesource of the virtual presence apparatus 12 to the client network node.In some of these examples, the server network node 42 administersconnections that relay data corresponding to communicant activity in thevirtual area from the client network node to the sink of the virtualpresence apparatus 12. In some examples, the virtual presence apparatus12 publishes data streams of different data stream types, and the servernetwork node 42 provisions the client network nodes to receive datastreams of different data stream types that are published by theparticular physical apparatus. In some examples, the server network node42 provisions a data stream connection between a client network node andthe virtual presence apparatus in response to a request from the clientnetwork node to subscribe to data published by the particular physicalapparatus. In some examples, the server network node 42 provisions adata stream connection between a client network node of a particularcommunicant and the virtual presence apparatus automatically upon entryof the particular communicant into the zone.

In some examples, the source of the virtual presence apparatus 12corresponds to a transducer that transforms human perceptible stimulusthat is broadcasted in the physical space into output data of therespective data stream content type. In some examples, the source of thevirtual presence apparatus 12 corresponds to a transducer thattransforms input data of the respective data stream content type intohuman perceptible stimulus that is broadcasted into the physical space.In some examples, the source of the virtual presence apparatus 12includes a microphone and the sink of the virtual presence apparatus 12includes a speaker. The microphone generates output voice data fromhuman voice sound projected into the physical space. The speakerprojects human voice sound into the physical space based on input voicedata associated with the virtual area. In some of these examples, theserver network node 42 administers connections that relay the outputvoice data from the apparatus to the client network node and that relaythe input voice data from the client network node to the apparatus. Insome examples, the source of the virtual presence apparatus 12 includesa camera that captures images of a scene in the physical space 14 andgenerates output image data from the captured images. In some of theseexamples, the server network node 42 administers a connection thatrelays the output image data from the virtual presence apparatus 12 tothe client network node. In some examples, the sink of the virtualpresence apparatus 12 includes a projector that projects images into thephysical space. In some of these examples, the server network node 42administers a connection that relays input control data for controllingthe projecting from the client network node to the virtual presenceapparatus 12. In some examples, the sink of the virtual presenceapparatus 12 includes a laser pointer that projects a laser beam intothe physical space. In some of these examples, the server network node42 administers a connection that relays input control data forcontrolling the projecting of the laser beam from the client networknode to the virtual presence apparatus 12.

Thus, in some examples, based on communicant input in connection withthe object representing the virtual presence apparatus 12, the servernetwork node 42 administers a connection between an audio source of theclient network node and an audio sink of the virtual presence apparatus12. In some examples, based on communicant input in connection with theobject, the server network node 42 administers a connection between anapplication sharing source of the client network node and an imageprojection sink of the virtual presence apparatus 12. In some examples,based on communicant input in connection with the object, the servernetwork node 42 administers a connection between a laser pointer controlsource of the client network node and a laser pointer control sink ofthe apparatus.

In some examples, the virtual presence apparatus 12 is located in aparticular physical space, and the server network node 42 locates theobject representing the virtual presence apparatus 12 in a particularone of the zones of the virtual area according to a mapping between theparticular physical space and the particular zone. In some of theseexamples, the mapping associates an identifier of the physical spacewith an identifier of the particular zone, creating a persistentassociation between the particular physical space and the particularzone of the virtual area. In some of these examples, the mappingadditionally associates an identifier of the physical apparatus 12 withthe identifier of the physical space. In some examples, thevisualization of the virtual area shows the particular zone with a labelthat connotes a name associated with the physical space.

In some examples, the server network node 42 establishes a respectivepresence in the virtual area for a particular communicant based on adetermination that the particular communicant is in the physical space14. In some examples, the server network node 42 receives location data(e.g., Global Positioning System (GPS) data) that is associated with theparticular communicant (e.g., by a GPS component of a mobile device,such as a mobile phone or other mobile communication device), anddetermines that the particular communicant is in the physical spacebased on comparison of the received location data with location dataassociated with the physical space. In some examples, the server networknode 42 receives audio data from the source of virtual presenceapparatus 12, and associates the audio data with a communicant in thephysical space based on comparison of the audio data with one or morevoice data records associated with respective communicants. The voicerecords typically correspond to voiceprints (also referred to as voicetemplates or voice models) that are created from features that areextracted from the recorded speech of known communicants in accordancewith a speaker recognition enrollment process. Each voiceprint isassociated with the identity of a particular communicant. The servernetwork node 42 typically associates the audio data with the communicantin response to a determination that features extracted from the audiodata correspond to the voiceprint previously associated with thecommunicant. In this way, the server network node 42 can automaticallyidentify communicants who are in the physical space without requiringthem to log into the network infrastructure service environment 44through respective client network nodes. Once a particular communicantin the physical space 14 has been identified, the server network node 42can automatically establish a presence for that communicant in thevirtual area associated with the virtual presence apparatus 12 and trackutterances from that communicant in the audio data captured by thevirtual presence apparatus such that visual cues indicative of the stateof that communicant's voice (e.g., speaking or silent) can be presentedin the spatial visualization of the virtual area that is displayed tothe remote communicant on the remote client network node 16.

FIG. 7 shows an example of a method that is performed by an example ofthe communications application 20 for communicating between a virtualarea and a physical space. In accordance with this method, thecommunications application 20 displays a graphical interface thatincludes a graphical representation of the virtual area that supportsestablishment of respective presences of communicants operatingrespective client network nodes, a graphical representation of each ofthe communicants who is present in the virtual area, and a graphicalrepresentation of an object associated with an apparatus (e.g., thevirtual presence apparatus 12) in the physical space (FIG. 7, block110). The apparatus has an apparatus sink that is complementary to theclient source and an apparatus source that is complementary to theclient sink. The communications application 20 establishes a respectiveconnection between each active pair of complementary sources and sinksof the client network node and the apparatus in association with thevirtual area, where each connection supports transmission of therespective data stream content type between the apparatus and the clientnetwork node (FIG. 7, block 112). The communications application 20presents interaction controls associated with the object for interactingwith communicants who are present in the physical space (FIG. 7, block114).

In some examples of the method of FIG. 7, the graphical representationof the virtual area corresponds to a virtualized representation of thephysical space. In some of these examples, the virtualizedrepresentation connotes the real-world physical space. For example, thevirtualized representation may have a virtual presentation thatresembles one or more distinctive visual features of the real-worldphysical space or the virtualized representation may include adescriptive name or other label that is associated with the real-worldphysical space.

In some examples, the communications application 20 receives from anetwork service administering the virtual area a specification fordisplaying the graphical representation of the object in spatialrelation to the graphical representation of the virtual area.

In some examples, the communications application 20 shows in thegraphical interface indications of respective states of the apparatussource of the virtual presence apparatus 12 in connection with thegraphical representation of the object. In some of these examples, theprocess of showing the state indications involves displaying a firstvisual cue when the virtual presence apparatus source is in an activestate, and displaying a second visual cue that is different from thefirst visual cue when the virtual presence apparatus source is in aninactive state.

In some examples, based on communicant input in connection with theobject, the communications application 20 establishes a connectionbetween an audio source of the client network node and an audio sink ofthe virtual presence apparatus 12.

In some examples, based on communicant input in connection with theobject, the communications application 20 establishes a connectionbetween an application sharing source of the client network node and animage projection sink of the virtual presence apparatus 12.

In some examples, based on communicant input in connection with theobject, the communications application 20 establishes a connectionbetween a laser pointer control source of the client network node and alaser pointer control sink of the virtual presence apparatus 12.

In some examples, the source of the virtual presence apparatus 12includes a microphone and the sink of the virtual presence apparatus 12includes a speaker. The microphone generates output voice data fromhuman voice sound projected into the physical space, the speakerprojects human voice sound into the physical space from input voice dataassociated with the virtual area, and the communications application 20establishes connections that relay the output voice data from thevirtual presence apparatus 12 to the client network node and that relaythe input voice data from the client network node to the virtualpresence apparatus 12.

In some examples, the source of the virtual presence apparatus 12includes a camera that captures images of a scene in the physical spaceand generates output image data from the captured images, and thecommunications application 20 establishes a connection that relays theoutput image data from the virtual presence apparatus 12 to the clientnetwork node.

In some examples, the sink of the virtual presence apparatus 12 includesa projector that projects images into the physical space, and thecommunications application 20 establishes a connection that relays inputcontrol data for controlling the projecting from the client network nodeto the virtual presence apparatus 12.

In some examples, the sink of the virtual presence apparatus 12 includesa laser pointer that projects a laser beam into the physical space, andthe communications application 20 establishes a connection that relaysinput control data for controlling the projecting of the laser beam fromthe client network node to the virtual presence apparatus 12.

FIG. 8 shows an example of a graphical interface 120 that is generatedby the communications application 20 on a client network node (e.g.,client node 16) for interfacing a user with an example 122 of thevirtual presence apparatus 12 in the physical space 14.

The graphical interface 120 includes a toolbar 124 and a viewer panel126. The toolbar 124 includes a headphone control 128 for toggling onand off the local speakers of the client network node, a microphonecontrol 130 for toggling on and off the local microphone of the clientnetwork node, and a view screen button 132 for setting the viewer panel126 to content being shared in connection with the view screen object60. The user also may select the view screen object 60 to initiate anapplication sharing session in the virtual area 46. The viewer panel 126typically shows communicant selectable content being rendered by theclient network node. Examples of such content include a spatialvisualization of the virtual area 46 (currently shown) and applicationcontent (e.g., web service content rendered by a web browser applicationsuch as Microsoft® Internet Explorer®, or document content beingrendered by a document processing application such as Microsoft® Word®or Power Point® software applications).

In the example shown in FIG. 8, the virtual presence apparatus 122 is avirtual area enabled speakerphone, which is represented by aspeakerphone object 138 in the spatial visualization of the virtual area46 that is shown in the viewer panel 126. The virtual presence apparatus122 includes a microphone that generates output voice data from humanvoice sounds projected into the physical space 14 and a speaker thatprojects human voice sounds into the physical space 14 based on inputvoice data associated with the virtual area. The “on” or “off” state ofthe speakerphone microphone is depicted in the spatial visualization ofthe virtual area by the presence or absence of a series of concentriccircles 140 that dynamically radiate away from the speakerphone object138 in a series of expanding waves. When the microphone is on, theradiating concentric circles 140 are present and, when the microphone isoff, the radiating concentric circles 140 are absent. In addition to oralternatively, the current activity state of the speakerphone microphonechannel is indicated by a dynamic visualization that lightens anddarkens the speakerphone object 138 in realtime to reflect the presenceor absence of audio data on the speakerphone microphone channel. Thus,the user can determine when a communicant in the physical space 14 isspeaking by the “blinking” of the coloration of the speakerphone object138.

FIG. 9 shows an example of a graphical interface 150 that is generatedby the communications application 20 on a client network node (e.g.,client node 16) for interfacing a user with an example 152 of thevirtual presence apparatus 12 in the physical space 14.

The graphical interface 150 includes the toolbar 124 and the viewerpanel 126 of the graphical interface 120 shown in FIG. 8. The toolbar124 includes the headphone control 128 for toggling on and off the localspeakers of the client network node, the microphone control 130 fortoggling on and off the local microphone of the client network node, anda view screen button 132 for setting the viewer panel 126 to contentbeing shared in connection with the view screen object 60. The user mayselect the view screen object 60 to initiate an application sharingsession in the virtual area 46. The viewer panel 126 typically showscommunicant selectable content that is rendered by the client networknode.

In the example shown in FIG. 9, the virtual presence apparatus 152 is avirtual area enabled device that integrates speakerphone and digitalprojector functionalities. The virtual presence apparatus 152 includes amicrophone that generates output voice data from human voice soundsprojected into the physical space 14, a speaker that projects humanvoice sounds into the physical space 14 based on input voice dataassociated with the virtual area, and a projector that projects lightinto the physical space 14 responsive to input data transmitted by theclient network node in connection with the virtual area 46.

The virtual presence apparatus 152 is represented by a projector object154 in the spatial visualization of the virtual area 46 that is shown inthe viewer panel 126. In this example, when the user selects theprojector object 154, the communications application 20 modifies thegraphical interface 150 to include a Share button 134 and a Stop button136 in the tool bar 124, and sets the viewer panel 126 to display thecontents of an application being shared. The user initiates anapplication sharing session in the physical space 14 by selecting theShare button 134. In response to the selection of the share button 134,the communications application 20 provides an interface that enables theuser to select an application to share (e.g., Microsoft® PowerPoint®),sets the viewer panel 126 to display the contents being rendered by theselected application, and streams screen share data to the virtualpresence apparatus 152, which projects the screen share data onto thereal-world view screen 34 in the physical space 14. Examples of systemsand methods of generating and streaming screen share data are describedin U.S. patent application Ser. No. 12/418,270, filed Apr. 3, 2009. Theuser can terminate the application sharing session in the physical space14 by selecting the Stop button 136.

FIG. 10 shows an example of a graphical interface 160 that is generatedby the communications application 20 on a client network node (e.g.,client node 16) for interfacing a user with an example 162 of thevirtual presence apparatus 12 in the physical space 14. In this example,the communicant 34 is giving a presentation on a white board 158 in thephysical space 14.

The graphical interface 160 includes the toolbar 124 and the viewerpanel 126 of the graphical interface 120 shown in FIG. 8. The toolbar124 includes the headphone control 128 for toggling on and off the localspeakers of the client network node, the microphone control 130 fortoggling on and off the local microphone of the client network node, anda view screen button 132 for setting the viewer panel 126 to contentbeing shared in connection with the view screen object 60. The user mayselect the view screen object 60 to initiate an application sharingsession in the virtual area 46. The viewer panel 126 typically showscommunicant selectable content being rendered by the client networknode.

In the example shown in FIG. 10, the virtual presence apparatus 162 is avirtual area enabled device that integrates a speakerphone, a digitalprojector, and a camera. The speakerphone includes a microphone thatgenerates output voice data from human voice sounds projected into thephysical space 14, and a speaker that projects human voice sounds intothe physical space 14 based on input voice data associated with thevirtual area. The projector projects light (e.g., images, shapes, lines,and spots) into the physical space 14 responsive to input datatransmitted by the client network node in connection with the virtualarea 46. In some examples, the projector is a digital image projector.In other examples, the projector is a remote-controlled laser pointer.The camera captures images of a scene in the physical space 14 (e.g.,images of the whiteboard 158) and generates output image data from thecaptured images. The camera may be implemented by any type of imagingdevice that is capable of capturing one-dimensional or two-dimensionalimages of a scene. The camera typically is a digital video camera.

The virtual presence apparatus 162 is represented by a projector-cameraobject 164 in the spatial visualization of the virtual area 46 that isshown in the viewer panel 126. In this example, when the user selectsthe projector-camera object 164, the communications application 20modifies the graphical interface 150 to include a Share button 134 and aStop button 136 in the tool bar 124, and sets the viewer panel 126 todisplay the images captured by the virtual presence apparatus 162 in thephysical space 14. The user initiates a sharing session in the physicalspace 14 by selecting the Share button 134. In response to the selectionof the Share button 134, the communications application 20 sets theviewer panel 126 to display the images captured by the virtual presenceapparatus 162, provides an interface that enables the user to provideinputs in relation to the images displayed in the viewer panel (e.g.,superimpose graphical content, such as predesigned or hand drawn imagesor comments, onto the images), and streams data describing the inputs tothe virtual presence apparatus 162, which projects the streamed dataonto the whiteboard 158 in the physical space 14. The user can terminatethe sharing session in the physical space 14 by selecting the Stopbutton 136.

In the illustrated example, the communications application 20 providesdrawings tools (e.g., the pencil tool 166) that allow the user tosuperimpose lines, shapes (e.g., the ellipse 168), and other graphicalcontent onto the image of the view screen 34 captured by the cameracomponent of the virtual presence apparatus 162.

In examples in which the projector component of the virtual presenceapparatus 162 is a digital projector, the communications application 20may stream data describing the user inputs to the virtual presenceapparatus 162. In examples in which the projector is a remote-controlledlaser pointer, the communications application 20 may convert the userinputs into control data for controlling the movement of theremote-controlled laser pointer in the physical space 14. In this way,the user can interact with the communicants in the physical space 14 ina substantive way. For example, the user can provide comments or othervisual indications that highlight or direct a viewer's attention tospecific parts of the presentation being given by the communicant 32 inconnection with the white board 158. In some examples, the graphicalinterface 160 includes additional controls for streaming applicationsharing data from the client network node to the virtual presenceapparatus 162 for projection onto the whiteboard 158 or other surface inthe virtual space 14, as described above in connection with the exampleshown in FIG. 9.

FIG. 11 shows an example of a graphical interface 170 that is generatedby the communications application 20 on a client network node forinterfacing a user with an example 172 of the virtual presence apparatus12 in the physical space 14. In this example, the communicant 34 isgiving a presentation on a white board 158 in the physical space 14.

The graphical interface 170 includes the toolbar 124 and the viewerpanel 126 of the graphical interface 120 shown in FIG. 8. The toolbar124 includes the headphone control 128 for toggling on and off the localspeakers of the client network node, the microphone control 130 fortoggling on and off the local microphone of the client network node, anda view screen button 132 for setting the viewer panel 126 to contentbeing shared in connection with the view screen object 60. The user mayselect the view screen object 60 to initiate an application sharingsession in the virtual area 46. The viewer panel 126 typically showscommunicant selectable content being rendered by the client networknode.

In the example shown in FIG. 11, the virtual presence apparatus 162 is avirtual area enabled device that integrates a speakerphone and a camera.The speakerphone includes a microphone that generates output voice datafrom human voice sounds projected into the physical space 14, and aspeaker that projects human voice sounds into the physical space 14based on input voice data associated with the virtual area. The cameracaptures images of a scene in the physical space 14 and generates outputimage data from the captured images. The camera may be implemented byany type of imaging device that is capable of capturing one-dimensionalor two-dimensional images of a scene. The camera typically is a digitalvideo camera.

The virtual presence apparatus 172 is represented by a camera object 174in the spatial visualization of the virtual area 46 that is shown in theviewer panel 126. In this example, when the user selects the cameraobject 174, the communications application 20 modifies the graphicalinterface 150 to include a View button 176 and a Stop button 178 in thetool bar 124, and sets the viewer panel 126 to display the imagescaptured by the virtual presence apparatus 162 in the physical space 14.The user initiates a viewing session in the physical space 14 byselecting the View button 176. In response to the selection of the Viewbutton 176, the communications application 20 sets the viewer panel 126to display the images captured by the virtual presence apparatus 172,provides an interface 180 that enables the user to control the view ofthe physical space that is captured by the camera component of thevirtual presence apparatus, and streams data describing the controlinputs to the virtual presence apparatus 172, which moves the camerabased on the streamed data. In the illustrated example, thecommunications application 20 provides a navigation control tool 180that allows the user to control the pan and tilt of the camera componentof the virtual presence apparatus 172. In this way, the remotecommunicant can interact with the physical space 14 in a substantive way(e.g., see different views of the persons and activities in the physicalspace 14). The user can terminate the sharing session in the physicalspace 14 by selecting the Stop button 178.

In some examples, different elements of the graphical interfacesdescribed above in connection with the examples shown in FIGS. 8-11 areincorporated into a single graphical interface that may be used tointeract with the virtual presence apparatus 164, which integrates aspeakerphone, a digital projector, and a camera. In these examples, thegraphical interface provides independent control over the respectivefunctionalities of the speakerphone, the digital projector, and camerato enable application sharing, image projection of comments and otherannotations, and camera viewing modes of operation.

III. CONCLUSION

Other embodiments are within the scope of the claims.

The invention claimed is:
 1. A method performed by one or moreprocessors executing one or more program instructions encoded in one ormore non-transitory structures, the method comprising: storingassociations between voiceprints of communicants and their respectiveidentities; receiving audio data transmitted by a device co-located in aphysical space with at least one of the communicants, wherein activitiesin the physical space are bridged into a particular virtual area;matching the received audio data to a respective one of the identitiescorresponding to a particular one of the communicants based oncomparisons of features of the received audio data with the voiceprints;automatically establishing a presence for the particular communicant inthe particular virtual area based on the matching, wherein the virtualarea corresponds to the physical space; transducing human perceptiblestimulus, including audio stimulus, between the virtual area and thephysical space to integrate audio interactions of the particular one ofthe communicants between the physical space and the virtual area;automatically establishing an audio communications channel between thedevice and the particular virtual area; and by the device in thephysical space, processing the audio data transmitted over the audiocommunications channel, and tracking utterances of the particularcommunicant in the physical space based on the processed audio data. 2.The method of claim 1, further comprising, by the device in the physicalspace, transducing sounds in the physical space into the audio data andtransmitting the audio data and an identifier of the device.
 3. Themethod of claim 2, further comprising locating the particularcommunicant in the physical space based on a mapping between theidentifier of the device and the physical space.
 4. The method of claim2, wherein the establishing comprises automatically establishing thepresence for the particular communicant in a particular zone of theparticular virtual area based on a mapping between the identifier of thedevice and the particular zone.
 5. The method of claim 1, furthercomprising, by the device in the physical space, transducing audio datareceived over the audio communications channel into sounds that arebroadcasted into the physical space.
 6. The method of claim 1, whereinthe virtual area supports establishment of respective presences ofcommunicants operating respective network nodes and supportscommunications between communicants who are in the virtual area.
 7. Asystem, comprising: a network node in a physical space configured totransduce sounds in a physical space into audio data and transmit theaudio data to a particular network address, wherein the network node isa device co-located in a physical space with at least a firstcommunicant and activities in the physical space are bridged into aparticular virtual area; and a network service platform associated withthe network address and configured to perform operations comprising:storing associations between voiceprints of communicants, including thefirst communicant, and their respective identities, receiving audio datatransmitted by the network node, matching the received audio data to arespective one of the identities corresponding to a particular one ofthe communicants based on comparisons of features of the received audiodata with the voiceprints, automatically establishing a presence for theparticular communicant in the particular virtual area based on thematching, wherein the virtual area corresponds to the physical space,transducing human perceptible stimulus, including audio stimulus,between the virtual area and the physical space to integrate audiointeractions of the particular one of the communicants between thephysical space and the virtual area; automatically establishing an audiocommunications channel between the device and the particular virtualarea; and by the device in the physical space, processing the audio datatransmitted over the audio communications channel, and trackingutterances of the particular communicant in the physical space based onthe processed audio data.
 8. The method of claim 7, further comprising,by the network node in the physical space, transducing sounds in thephysical space into the audio data and transmitting the audio data andan identifier of the network node.
 9. The method of claim 8, furthercomprising locating the particular communicant in the physical spacebased on a mapping between the identifier of the network and thephysical space.
 10. The method of claim 8, wherein the establishingcomprises automatically establishing the presence for the particularcommunicant in a particular zone of the particular virtual area based ona mapping between the identifier of the network node and the particularzone.
 11. The method of claim 7, further comprising, by the network nodein the physical space, transducing audio data received over the audiocommunications channel into sounds that are broadcasted into thephysical space.
 12. The method of claim 7, wherein the virtual areasupports establishment of respective presences of communicants operatingrespective network nodes and supports communications betweencommunicants who are in the virtual area.